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Professional paper 
ABSTRACT 

Background: Nowadays, general practitioners 
(GPs) and family doctors (FDs) face increas- 
ing demands, as a consequence of complex 
patients' expectations, developments in science 
and technology, and limitations within health- 
care systems which can result in competency 
gaps. Therefore, there is a need to identify which 
competencies in quality improvement (01) are 
most important for GPs and FDs to possess in 
order to meet the demands of contemporary 
health care practice. To date, however, little 
information is available on the self-assessment 
of competencies related to 01 among GPs and 
FDs. To deal with these issues, a project on 01 
in continuous medical education was launched 
in 2011. The project aims to broaden the GPs'/ 
FDs' continuous education offer, its quality and 
attractiveness, as well as provide them with 



opportunities for vocational advancement and 
enable the development of common, European 
frame of reference for GPsVFDs' occupational 
competencies. The third work package of the 
project consists of the validation research of 
the questionnaire developed on the basis of 
the competency framework in 01 for GPs/FDs 
in Europe. Methods: A cross-sectional study 
will be carried out usingthe self-assessment 01 
questionnaire which was originally developed 
in English and subsequently it was cross- 
culturally adapted in Slovenian, Albanian and 
Polish settings by use of a pilot study on a 
conveniently selected group of FDs/GPs ( N= 10 ) 
in each participating country. The final version 
of the questionnaire will be administered to 
large samples in each country involved in the 
survey. Two weeks after the first administra- 
tion of the questionnaire, a second round, with 
the same procedure and including the same 



group of respondents, will follow. Psychomet- 
ric tests will be conducted including internal 
consistency (after the initial and subsequent 
application of the instrument) and stability 
overtime (two-week test-retest reliability). 
Discussion: This self-assessment study will 
demonstrate the complex environment in which 
general practice/family medicine operates 
and, eventually, this gap analysis will set out 
strategically important areas for collaborative 
efforts related to 01 in primary care. The authors 
consider that the study should be extended to 
other European countries to help identify most 
required competencies that GPs/FDs should 
possess in Europe and thus stir system and 
educational debate around 01 curricula and 
training for primary care in Europe. 
Key words: competencies, family physi- 
cians, quality improvement, reliability, self- 
assessment questionnaire, validation study. 



1. BACKGROUND 

Modern medicine recognises 
that the outcomes of clinical care 
not only depend on how doctors 
put their clinical knowledge and 
training into practice, but also on 
other skills such as being able to deal 
with the continuous flow of new in- 
formation, emerging medical evi- 
dence and effective management of 
available resources (1). Today, family 
doctors (FDs) face increasing de- 
mands, as a consequence of complex 
patients' expectations (2), develop- 



ments in science and technology, 
and limitations within healthcare 
systems (3) which can result in com- 
petency gaps. This means that FDs 
of today need to be able to reflect on 
the organizational systems in which 
they deliver care and, as needed, ef- 
fectively participate in changing 
those systems to improve the quality 
of care (4). Quality improvement 
has been recognized as an impor- 
tant part of continuous medical 
education (CME) by several impor- 
tant parties in education and quality 



since at least 2001. Therefore, there is 
a need to identify which competen- 
cies in quality improvement (QI) are 
the most important for general prac- 
titioners (GPs) and FDs to possess in 
order to meet the demands of con- 
temporary health care practice. Var- 
ious assessment tools of good quality 
(5) have been developed to assess 
doctors learning processes and their 
competences at the authentic work- 
place (6). Little information is avail- 
able on the self-assessment of com- 
petencies related to QI among pri- 
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mary care physicians, family medi- 
cine (FM) teachers and primary care 
patients. 

To deal with the above issues, a 
project on QI in CME was launched 
in 2011. 

1.1. Introduction to the project 

GPs' and FDs' involvement in 
the cardiovascular diseases (CVD) 
"first-line" treatments, prevention 
and health promotion is a problem 
of the highest importance in Eu- 
ropean public health (PH) (7, 8). 
First stages of the treatment of pa- 
tients with chronic diseases (espe- 
cially, CVD) might be improved by 
new approaches to lifelong learning 
(LLL) of GPs and implementing 
modern information technology 
(IT) in existing educational systems 
in Europe (9). This study is a part of a 
large European project entitled "In- 
novative lifelong learning of Euro- 
pean General Physicians in Quality 
Improvement supported by infor- 
mation technology" (InGPinQI) No. 
2010-1-PL1-LEO05-11473 financed 
by European Commission Lifelong 
Learning Program which consists 
of ten work packages (WPs) (Figure 
1). The InGPinQI project is focused 
on the improvement of the QI edu- 
cation in hypertension and diabetes 
mellitus, which are important prob- 
lems of public health in Central and 
Eastern Europe (10, 11, 12, 13). It aims 
to improve the existing continuous 
professional development (CPD) 
programs for GPs/FDs and teachers 
in family medicine and their compe- 
tencies in the field of QI. This might 
be achieved by implementing inno- 
vative tools (e.g. a web-based tool 
measuring the required competen- 
cies and educational needs, a dis- 
tance-learning course for GPs/FDs) 
in existing educational systems in 
Europe supported by IT (Figure 2). 

2. AIM AND OBJECTIVES 

The project aims to broaden the 
GPs'/FDs' educational offer and its 
quality and attractiveness, as well as 
provide GPs/FDs with opportuni- 
ties for vocational advancement and 
enables developing of common, Eu- 
ropean frame of reference for GPs'/ 
FDs' occupational competencies. 

The third work package (WP 3) 



of the project consists of the valida- 
tion research of the developed ques- 
tionnaire on the basis of the compe- 
tency framework in QI for GPs/FDs 
in Europe. The development of the 
framework has been extensively de- 
scribed elsewhere (14). The WP 3 has 
two main objectives: 

• To validate the QI self-assessment 
questionnaire and; 

• To identify the competency gaps be- 
tween currently possessed compe- 
tencies by GPs/FDs and those which 
are most required to be possessed by 
them to provide high level patient- 
centred care in Slovenian, Polish 
and Albanian health care context. 
Emphasis is placed on: 

• Current self-perceived level of 
competencies related to QI pos- 
sessed by GPs/FDs in Poland, 
Slovenia and Albania and, addi- 
tionally, perceived level of GPs/ 
FDs' competencies from the per- 
spective of patients (in Albania 
only). 

• The desirable/expected level of 
QI competencies that GPs/FDs 
should possess measured from 
the perspective of FM teachers 
in Slovenia, Poland, and Albania 
and, additionally, from the per- 
spective of the decision-makers 
(in Albania only). 

In addition, the gap or mismatch 
between the currently possessed 
competencies and the required ones 
is being identified and recommenda- 
tions are being made on the develop- 
ment and introduction of the educa- 
tional interventions in QI for GPs/ 
FDs. 

The aim of this article is to present 
the study protocol on the validation 
of the QI self-assessment question- 
naire. Additional aim is to get first 
results on the self-perceived level of 
QI competencies from the perspec- 
tive of the GPs/FPs, FM teachers, pa- 
tients and policy makers which will 
to identify priority areas within the 
general/family practice environment 
in Albanian, Slovenian and Polish 
context. 

2.1. Procedures of the validation 
of the QI self-assessment 
questionnaire 

QI self-assessment questionnaire 
The questionnaire is based on the 



QI Competency framework for GPs/ 
FDs in Europe (14). It consists of two 
parts: 1) assessment of competen- 
cies, and; 2) demographic informa- 
tion which varies depending on the 
target group of respondents who are 
GPs/FDs, FM teachers and trainers, 
patients and policy makers. 

Part 1 of the questionnaire in- 
cludes 6 comprehensive and inter- 
disciplinary competency domains: 
Patient Care & Safety, Effectiveness 
& Efficiency, Equity & Ethical Prac- 
tice, Methods & Tools, Leadership 
8c Management, and Continuing 
Professional Education. Thirty- 
five specific competencies are orga- 
nized into the 6 domains. We used 
a five-stage Dreyfus scale (15) which 
proved useful when thinking about 
medical education (15). It consists of 
five categories: novice (1 point), ad- 
vanced beginner (2 points), compe- 
tent (3 points), proficient (4 points), 
and expert (5 points). The summary 
score of the whole questionnaire 
ranges from a minimum 35 points 
to a maximum 180 points. The min- 
imum and maximum summary 
scores of the six domains range as 
follows: Patient Care & Safety (8-40 
points), Effectiveness & Efficiency 
(7-35 points), Equity & Ethical Prac- 
tice (8-40 points), Methods & Tools 
(5-25 points), Leadership & Manage- 
ment (4-20 points), and Continuing 
Professional Education (5-25 points). 

The second part of the question- 
naire consists of: 

• For GPs/FDs: gender, age, type of 
practice, whether it is public or 
private entity, the catchment ar- 
ea of the practice (number of in- 
habitants and rural/urban), num- 
ber of years served in practice, re- 
muneration system and whether 
they are involved in teaching ac- 
tivities; 

• For FM teachers/trainers: age, 
the catchment area of the prac- 
tice (number of inhabitants and 
rural/urban), years of experience 
as family medicine teachers, spe- 
cialization, type/level of training 
provided, position in education; 

• For patients: gender, age, current 
life situation, education status, 
number of times of seeing their 
family doctor in the period of 
past 12 months; 
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• For decision-makers: age, place 
of residence (number of inhab- 
itants and rural/urban), sector 
they work for, job title. 

2.2. Study design 

Three centres will be involved in 
the validation study: Institute for 
Development of Family Medicine 
with cooperation of Departments 
of Family Medicine Maribor and 
Ljubljana Medical School, Slovenia 
(SL), College of Family Physicians 
in Poland with cooperation with the 
Department of Family Medicine, Ja- 
giellonian University Medical Col- 
lege (PL) and Department of Public 
Health, University of Tirana, Al- 
bania (AL). 

The study aims to validate the 
questionnaire and to identify the 
existing gaps between the current/ 
self-perceived level of competencies 
related to QI possessed by GPs/FDs 
and the desirable/expected level of 
competencies by FD teachers FDs 
should possess to provide high level 
patient-centred care. In order to do 
so, several stages of the study are 
foreseen. 

Stages A & B of the study will 
constitute of a validation exercise as 
described below: 

a) The current/self-perceived level 
of competencies will be measured 
from the perspective of GPs/FDs in 
Poland and Slovenia and from the 
perspective of GPs/FDs and patients 
in Albania by using a self assess- 
ment QI questionnaire developed in 
the framework of InGPinQI Project 
(Figure 1). 

b) The desirable/expected level 
of competencies will be measured 
from the perspective of FM teachers/ 
trainers in Slovenia and Poland and 
from the perspective of FM teachers/ 
trainers and decision-makers in Al- 
bania. 

2.3. Validation Methods 

This cross-sectional study will be 
carried out as postal survey using 
the self-assessment QI questionnaire 
which has been translated back and 
forth from English to Slovenian, Al- 
banian and Polish and piloted on a 
conveniently selected group of FDs/ 
GPs (N=10) in each participating 
country (and patients in Albania). 




WP 3 — Development of the Competency Framework in QI for GPs in Europe and 
a web-based tool measuring educational needs and gap competencies on QI in FM 
supported by a validation study. 



WP 4 - Development of the 
Guide Book 



WP 5 -Design of the 
vocational training (VET) for 
teachers in FM 



WP 6 - Development of the 
Internet course dedicated to QI 
teachers in FM 



WP 7 - Development of new guidelines for the management of arterial hypertension and diabetes 
mellitus - a Polish perspective 




Figure 1. Overview the European project inGPinQI No. 2010-1-PL1-LE005-11473. The training materials 
(WP4, WP5, WP7) and the courses (WP5, WP6) developed in the project are based on the data of the 
identified gaps in the educational systems (WP1, WP2) and required GPs competencies (WP3) relatingto the 
vital problems in the European communities, namely: (a) inequalities in health and social determinants of 
health, (b) growing patient expectations towards quality of services, (c) fragmentary GP's education on 01, 
(d) unused expert knowledge of physicians, (e) growing demand on cross-border services. 



Furthermore, in three countries, the 
respective versions of the question- 
naires have been language and cul- 
turally adapted (16, 17, 18). We will 
adapt the questionnaire based on 
the suggestions from the pilot study. 

The final version of the question- 
naire will be then sent to the study 
group, described below. To assure 
the adequate response rate, the first 
reminder will be sent one week after 
the first round. Two weeks after the 
first questionnaire will be sent, the 
second round with the same proce- 
dure will follow. In case of no re- 
sponse, the researchers will try to 
contact the respondents by phone 
if the conditions allow. In case of 
too low response rate, focus groups 
will be arranged with a group of re- 
spondents which does not reach a re- 
quired quota. 

The study will be carried out in 
stages in Slovenia, Poland and Al- 
bania. For stage A & B, 2 psycho- 
metric tests will be used: internal 
consistency (after the initial applica- 
tion of the instrument) and stability 
over time (after reapplication of the 
test after 2 weeks to the same group 
of respondents) [see "Statistical anal- 
ysis" section for further details]. 

2.4. Ethical considerations 

The study will be performed in 
line with the Helsinki Declaration 



using survey. The participants will be 
informed about the goal and objec- 
tives of the study and will be given a 
possibility to opt out without giving 
explanation if by any chance they feel 
uncomfortable with the content of 
the questions. The information about 
the research will be posted together 
with the questionnaire. In case of no- 
response, the researchers will seek to 
get a proper consent from the respon- 
dents to contact them by the phone. 
If the focus group has to be arranged 
with a group of respondents which 
does not reach a required quota, an 
informed consent form will be de- 
veloped to be signed by the respon- 
dents. The ethical approval has been 
applied for and granted by: the Na- 
tional Ethics Committee in Slovenia 
(No. 96/05/12), the National Com- 
mittee of Biomedical Ethics in Al- 
bania. In Poland the study will be 
conducted under the terms of all rel- 
evant local and national legislation. 
Approval by Ethical Committee is 
not necessary because the study has 
not experimental design and will not 
involve patients. Moreover, protocol 
conforms to the ethical guidelines of 
the 1975 Helsinki Declaration. 

Stage A: Current/self-perceived 
level of QI competencies self-assess- 
ment (GPsVFDs and patients' per- 
spective 

Sampling 
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Report on the existing training 
programms, methods and tools Ql 
Report on the successfully implemented Ql 
projects concerning CVDs in Europe 
Report of educational needs and required 
competencies of GPs in Ql 
New guidelines for the management of 
arterial hypertension and diabetes 
mellitus: a Polish perspective 



Web-based tool to identify individual, 
educational needs in Ql 
GuideBook containing guidelines on 
effective development and implementation 
of Ql programmes on EU level 
Distance-learning module on Ql 



Innovative VET course on Ql 



Learning materials for VET course on Ql 



Figure 2. Tangible products developed in the inGPinQI project No. 2010-1-PL1- LE005-114P3 



The respondents for Stage A: GPs/ 
FDs and patients (only in Albania) 
will be randomly selected based 
on the general country register for 
GPs and GP practice records (for 
patients). In Poland, the database 
of the College of Family Physicians 
will be used. In Slovenia, the data- 
base of the Slovenian Family Medi- 
cine Society will be used. We aim to 
achieve N=100 responses from GPs/ 
FDs (and N=100 patients in Albania). 
In the pre-test phase, a convenient 
sample of 50 FDs and 100 patients 
will be recruited in Tirana only. 

After the validation exercise, the 
actual survey will include 2000 pa- 
tients in 4 districts of Albania (Ti- 
rana, Shkoder, Elbasan and Vlore). 
The sample will be randomly se- 
lected based on the GPs' lists in 
each of the districts included in the 
survey. As for the GPs, the ques- 
tionnaire will be administered to a 
random sample of GPs operating in 
primary health care centres in the 4 
districts included in the survey. 

The details of the study in stage A 
in each centre are presented below: 

Pre-test: Application of the ques- 
tionnaire on self-perceived level of Ql 
competencies from GPs/FDs perspective 
(N=100): 

• Calculation of internal consistency; 

• Reapplication of the questionnaire 
to the same groups of respondents 
after 2 weeks (calculation of test-re- 
test reliability). 

Stage B: Expected/required level 
ofQI competencies possessed by GPs 



from FM/GP teachers and decision 
makers perspective 
Sampling 

The respondents for Stage B:FM/ 
GP teachers and decision makers 
will be randomly or conveniently se- 
lected based on the general country 
register for GP/FD teachers and 
professional contacts for decision 
makers. In Slovenia, the database 
of the Slovenian Family Medicine 
Society will be used. We aim to 
achieve N=50 responses from GP/ 
FD teachers and N=50 from decision 
makers in respective centres. 

In Poland in order to select the 
randomized sample of FM teachers, 
approx. 10 centres providing edu- 
cational programmes for GPs/FDs 
(from the total number of 50 insti- 
tutions) will be randomly selected 
and invited to the study (a phone 
call with directors). If the response 
with a positive attitude will be in- 
sufficient, the sample will be com- 
plemented by a further number of 
randomly chosen institutions. 3-4 
questionnaires will be sent by a reg- 
ular mail to those centres which will 
be interested in participating in the 
study. Directors of those centres will 
indicate the respective teachers who 
should be involved in the study. If 
the response rate is insufficient they 
will be contacted by phone. 

The details of the study in stage B 
in each centre are presented below: 

Pre-test: Application of the question- 
naire on expected level of Ql competen- 
cies in GPs from FM/GP teaches per- 



spective (N-S0): 

• Calculation of internal consistency; 

• Reapplication of the questionnaire 
on the same groups of respondents 
after 2 weeks (calculation of test-re- 
test reliability) 

2.5. Statistical analysis 

Cronbach's alpha will be used to 
calculate the internal consistency of 
the questionnaire on self-perceived 
level of Ql competencies from GPs/ 
FDs perspective (stage A of the vali- 
dation study) and of the question- 
naire on expected level of Ql com- 
petencies in GPs/FDs from FM/GP 
teaches perspective (stage B of the 
study) .Cronbach's alpha will be cal- 
culated for the whole questionnaire 
and for each of the six domains. 

Spearman's who will be used to 
measure test-retest reliability (sta- 
bility over time) of the questionnaire 
on self-perceived level of Ql com- 
petencies from GPs/FDs perspec- 
tive (stage A of the validation study, 
during which the instrument will be 
reapplied after 2 weeks to the same 
group of respondents) and of the 
questionnaire on expected level of 
Ql competencies in GPs/FDs from 
FM/GP teaches perspective (stage B 
of the study, during which the in- 
strument will be reapplied after 2 
weeks to the same group of respon- 
dents). The test-retest reliability of 
the summary score will be calcu- 
lated for the whole questionnaire 
and for each of the six domains. 

3. EXPECTED RESULTS AND 
DISCUSSION 

With the planned study, we will 
validate the newly developed compe- 
tency framework questionnaire (19) 
in Polish, Albanian and Slovenian 
language. We expect that the ques- 
tionnaire will have good reliability 
and time stability. Based on this ques- 
tionnaire, we will also get first data 
on the self-perceived level of Ql com- 
petencies from the perspective of the 
GPs/FPs, FM teachers, patients and 
policy makers specifically focusing 
on the validity and reliability of the 
used instrument. This will enable us 
to identify priority areas within the 
general practice/family medicine en- 
vironment in Albanian, Slovenian 
and Polish context. We expect the 
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study will point to differences be- 
tween the self-perceived competen- 
cies by GPs/FDs and between the 
desired competencies by FD teachers 
and trainers. Based on this gap, we 
will be able to identify the weak- 
ness which could affect aspects of 
the quality of care being provided. 
Therefore, this study can contribute 
to the development of recommenda- 
tions for priority actions related to 
these gaps. Identification of the gaps 
between possessed and expected/re- 
quired level of competencies based 
on the results per centre will enable 
us to get a first idea on the recom- 
mendations for educational inter- 
ventions needed in QI education/ 
training for GPs/FDs. The results 
of the study will allow comparisons 
among participating countries with 
acknowledging specificities of dif- 
ferent health care systems (20, 21). 

This self-assessment study will 
demonstrate the complex environ- 
ment in which general practice/ 
family medicine operates and this 
gap analysis will set out strategically 
important areas for collaborative ef- 
forts related to QI in primary care. 
The authors believe that the study 
should be extended to other Euro- 
pean countries to help identify most 
required competencies that GPs/FDs 
should possess in Europe and thus 
stir system and educational debate 
around QI curricula and training for 
primary care in Europe. 
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CME: continuing medical educa- 
tion; CVD: cardiovascular diseases; 
FDs: family doctors; FM: family 
medicine; GPs: general practitio- 
ners; InGPinQI: Innovative lifelong 
learning of European General Phy- 
sicians in Quality Improvement sup- 
ported by information technology 
project; IT; information technology; 
LLL: lifelong learning; QI: quality 
improvement; WP: work package. 
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